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Abstract 


We  consider  Lagrange  dual  problems  in  which  the  vector  of  Lagrange 
multipliers,  in  addition  to  the  usual  nonnegativity  conditions,  satisfies 
certain  linear  constraints.  Of  course,  the  maximum  dual  value,  for  such 
restricted  classes  of  multipliers,  need  not  equal  the  primal  value,  even 
if  various  "constraint  qualifications"  are  appended. 

We  obtain  expressions  for  this  maximum  "constrained  dual"  value,  in 
terms  of  perturbation  functions  for  the  primal  convex  program,  and  value 
functions  associated  with  tht  linear  constraints.  At  least  some  infor- 
mation on  the  primal  perturbation  functions  appears  to  be  necessary  in 
order  to  put  upper  or  lower  bounds  on  the  "constrained  dual"  value.  For 
the  case  that  the  constraints  are,  simply,  that  a positive  weighted  sum 
of  the  multipliers  shall  not  exceed  some  given  bound  (plus,  of  course,  the 
usual  nonnegativities),  only  two  values  of  an  associated  one-dimensional 
perturbation  function  is  needed  to  obtain  such  upper  and  lower  bounds. 
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Dedicated  to  Dick  Duffin 


Let  S be  a nonempty  convex  subset  of  a vector  space  and  gk:  S -*  R 

be  convex  (not  necessarily  continuous)  functions.  An  ordinary  constrained 
convex  optimization  program  (see  e.g.,  [7  , Section  28])  is 


minimize  f(x) 

(P)  subject  to  g±(x)  < 0 for  1 < i < k 

and  x e S . 

Throughout  the  paper,  we  assume  that  (P)  is  consistent. 
For  any  \j,...,Xk  > 0 the  Lagrange  dual  problem  is 


minimize  f(x)  + 2 Xjg^x) 

i=l 

subject  to  x c S . 


The  value  v(D^)  (D^)  clearly  does  not  exceed  the  value  v(P)  of  (P) 

and  it  may  be  - <*> . One  uses  the  unconstrained  optimization  problem  (D..  ) to 

A. 

give  information  about  the  constrained  one  (P)  . If  v(D)  « sup  vflL  ),  then 

x>o  ^ 

v(D)  < v(P). 

In  this  paper,  we  study  (D^ ) for  those  X > 0 satisfying  certain 
conditions,  for  example: 


2 X < M; 
1 


X.  < M,  1 < i < k; 


or,  more  generally, 
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AX  < b , where  b * R™  and  A e r”™^  . 


2 


\ 


In  other  words  if  T is  a finite  system  of  linear  constraints  on  the  A ^ , 

We  are  interested  In  v(DT)  = sup  {.v(D^)  | A satisfies  '2'}. 

We  establish  relations  between  v(DT)  and  certain  relaxations  of  (P). 

In  the  spirit  of  Duff  in  [4],  Blair  [ l],  Charnes,  Cooper,  and  Kortanek 
t 3 ] , Duffin  and  Jeroslow  [ 5 ] , and  Duff in  and  Karlovitz  [ 6 ] , we  will 
use  theorems  about  systems  of  linear  inequalities  systematically. 

We  begin  with  the  situation  in  which  {x  | AX  < b , X > 0}  is  bounded 
(i.e.,  a polytope).  This  case  includes  (1)  and  (2)  above;  and  has 
simpler  conclusions  than  the  case  of  arbitrary  A,  which  we  treat  later. 

Section  1:  (X  | AX  < b , X > 0j  is  bounded 

Lemma  1;  Let  P = (x  ( AX  < b,  X > 0}  be  nonempty  and  bounded  (i.e.,  a polytope). 
There  is  no  X e P such  that 

k 

(1)  f(x)  + Z Xtgt(x)  > L 

for  every  xeS,  if  and  only  if,  there  is  a single  point  xeS  such  that  the  finite 
system  of  linear  inequalities  in  unknowns  X.,...,X^ 

AX  < b 
k 

(S  ) 2 gt(x)Xt  > L - f(x) 

- i»l  1 1 

X > 0 


alone  has  no  solution. 
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Proof:  The  "if"  part  is  immediate.  We  now  prove  the  "only  if"  part. 

Let  xeS  and  denote,  by  Tx  c Rk,  the  set  of  solutions  to  Sx-  Tx  is 

compact  for  every  x,  since  it  is  closed  by  virtue  of  being  a polyhedron, 

and  bounded  as  it  is  a subset  of  P.  Now  (1  T = 0 if  (1) 

xeS 

fails.  Therefore,  there  is  a finite  F C S such  that  D T is  empty,  by 

xcF 

standard  compactness  results.  Then  the  system  of  linear  inequalities 


AX  < b 
k 

(2)  Z g.  (x)X,  > L - i(x),  for  all  xeF 

i=l 

X > 0 


has  no  solution. 

By  the  Kuhn-Fourier  Theorem  [ 8 ] there  are  nonnegative  w e Rm , ux  e R 

such  that  wfcA  - Z u (g  (x) , . . . ,g,  (x))  > 0 and  wCb  + Z u (f(x)  - L)  < 0. 
xeF  X 1 * x«F 

Since  P is  nonempty  u > 0 for  at  least  one  xeF.  Let  N - Z u > 0, 

x xeF 

and  x = - Z u x.  Since  FES,  clearly  x«S.  By  convexity, 

— N _ x 
xeF 

f(x)  < J Z uf(x)andg.(x)<J  Z u g (x).  Therefore 
N x«F  x 1 NxeFxl 

wfcA  - N(g1(x),...,gn(x))  and  vh  + N(f(x)  - L))  < 0,  i.e,,  the  system 


(S  ) has  no  solutions. 
' x 


Q.E.D. 


Theorem  1:  Let  P be  as  in  the  lemma.  Let  V:  R -•  R be  defined  by: 


(3)  V (u)  = max  {uX  | AX  < b,  X > 0}. 

V 

Let  p:  R -*  R be  defined  by: 


(4) 


p(u)  = inf  {f  (x)  | g^(x)  < u^  for  1 < i < k and  x eS}, 
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where  we  set  p(u)  = + « if  there  is  no  x e S satisfying  gi(x)  < ^ for 
1 < i < k,  and  r + ® = + ® for  all  reals  r «R.  Also,  define 

(5)  L = inf  {v (u)  + p(u)}. 

„k 

ucR 

Xf  l * - ® , then  v(D.  ) = - ® for  all  X satisfying  X > 0 and  AX  < b. 

K 

In  fact,  if  L c R U then  v(D^)  < L for  all  X with  X > 0,  AX  < b. 

Moreover,  there  exists  X > 0 with  AX  < b and  v(D^)  = L. 

Proof : First,  suppose  that  LeR. 

Let  X e P.  We  show  that  v(D^)  < L.  In  fact,  if  v(D^)  > L,  then 
for  some  e > 0, 

k 

(6)  f(x)  + E X.g, (x)  > L+e  for  all  xcS. 

i=l 

Let  u«Rk  be  arbitrary.  If  p(u)  ^ ® , then  p(u)  sR,  since  p(u)  » - ® 

is  ruled  out  by  L e R (note  that  V(u)  eR  for  all  ueR  , by  inspection  of 
(3)).  For  any  xeX(u)  = [x«S  | g^x)  < u£  for  1 < i < k}  / 0,  (6)  gives 

k 

(6) f  f (x)  + E X.u,  > L + e 

i=l 

as  all  X±  > 0.  From  (6)' , it  follows  at  once,  by  taking  the  infimum  over 
x eX(u),  that 

k 

(7)  p(u)  + E X.u  > L + e. 

i-1 

Then  invoking  the  definition  (3)  of  V(u),  we  have 

(8)  P(u)  + V (u)  > L + e. 

Ic 

Also,  if  p(u)  =+®,  (8)  is  true.  Therefore,  (8)  holds  for  all  ueR  , 
contradicting  the  definition  (5)  of  L. 
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Continuing  with  our  assumption  LcR,  note  that,  for  any  xeS, 

(9)  L - f(x)  < L - p(g1(x),...,gk(x)) 

5 V (g^ (x) , • • . ,gk(x) ) , 

using  u = (g1(x), . .. ,gk(x))  in  the  definition  (5)  of  L.  From  (9),  (Sx) 
has  a solution.  By  Lemma  1,  a X,  as  described  in  the  Theorem,  exists. 

Finally,  suppose  that  L *•-<*»  . Then  we  repeat  the  discussion,  in 
equations  (6)-(8),  with  any  finite  value  L/  replacing  L + e on  the  r.h.s. 
in  (6).  Again,  we  obtain  a contradiction.  This  proves  that  v(D^)  * - ® 
if  X > 0 and  AX  < b . 

Q.E.D. 

The  function  p above  is  the  usual  perturbation  function  [ 7 ] . V is  the 
criterion  "value  function"  associated  with  a linear  program,  and  as  is 
well  known,  it  can  be  written  as  the  maximum  of  finitely  many  linear 
affine  functions,  each  of  which  corresponds  to  an  extreme  point  of  the 
feasible  region. 

Theorem  1 tells  us  how  good  a value  we  can  hope  for  in  the  program 

(D.  ) given  that  the  X must  lie  in  P.  Consider  a family  of  polytopes 

K 

= {X  | AX  < ab  , X> 0}  for  a > 0,  b eRm,  b >0.  As  a increases, 
becomes  larger.  Given  L we  can  ask  what  the  smallest  a is  such  that  there 

exist  X «P  with  v(D.  ) > L.  This  is  a "converse"  of  the  question  answered 

Of  A. 

by  Theorem  1. 

Theorem  2:  Let  P^  be  polytopes  as  above.  Define  a "generalized  pertur- 
bation function"  G:  R™  -*  R,  depending  on  A,  by 

(10)  G(w)  - inf  (f(x)  | subject  to  xeS  and  (g^x) , . . . ,gk(x))  < wCA}. 
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Define  a "modulus"  for  LcR,  by: 

(11)  h.  (A,L)  - sup  L ' , 

b w 0 w b 

0 a 

where  we  conventionally  assign  ^ = 0,  -(-  ® ) ■ + ® , L+®  ■ + ® , q = - “ 

for  a < 0,  = + ® for  a>0,  L-®=-®,  and  we  set  G(w)  = + ® if  there 

m 

exists  no  x c S satisfying  g . (x)  < 2 w.a,.  for  i < 1 < k. 

1 j=l  J 3 

Suppose  that  a > 0,  a «R,  b > 0. 

Then  there  exist  X e such  that  v(D^)  > L if  and  only  If  or  > h^ (A,L) . 

(In  particular,  if  h^(A,L)  = + ® , then  L is  not  obtainable  for  any  a;  and 
consequently  also,  v(D)  < L if  b > 0.) 

Proof : First  we  establish  the  "only  if"  part. 

Suppose  f(x)  + 2 X^g^(x)  > L for  all  xeS,  i.e.,  v(D^)  > L,  with 

X eP  . Let  w > 0.  Choose  e > 0 and  let  x be  such  that,  assuming  G(w)  finite, 
or  — o 

(g1(xQ), . . . ,gn(xQ))  < wfcA  and  f(xQ)  < G(w)  + e.  Since  X eP^, 

L - G(w)  - e < L - f(xQ)  < 2 Xigi(xQ)  < wCAX  < ^(wS).  Since  e was 
arbitrary,  L-G(w)  <Q'(wtb).  The  same  result  holds  if  G(w)  = + •;  and 
the  same  reasoning  shows  that  G(w)  = - ® cannot  occur  (as  v(D^)  = - “for  all  X > 0). 

If  w*!)  jt  0,  then  wH  > 0 (as  w > 0 and  b > 0),  so  (L  -G(w))/(wtb)  < a. 

If  w*!)  = 0,  we  have  L < G(w),  and  so  by  our  conventions  (L  -G(w))/(wtb)  < a 
as  a > 0.  Thus  h^A.L)  < a,  as  w > 0 was  arbitrary. 

We  now  establish  the  "if"  part.  Let  a > h^A.L).  We  show  the  system 
(Sx)  , with  b replaced  with  orb,  has  solutions  for  all  xeS  and  apply  Lemma  1. 

Let  x«S.  By  the  Kuhn-Fourier  Theorem,  and  the  fact  that  P^  is 
nonempty,  (S  ) has  no  solutions  only  if  there  is  a w > 0 such  that 
wCA  > (g1(x),...,gk(x))  and  o rwS  < L - f(x).  Since  L - f(x)  < L - G(w), 
we  obtain  owS  < L - G(w).  If  w*b  > 0,  we  have  a < (L  -G(w))/wtb,  which 
contradicts  a > h^A.L).  If  » 0,  we  have  G(w)  < L,  hence 

t ~~  . 
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(L-G(w))/wtb  = + » by  our  conventions.  Therefore,  h^(A,L)  = + 00  » again 
contradicting  or  > h^A.L). 

Q.E.D. 


For  the  case  where  some  components  of  b are  negative  there  is  the 
result: 


Theorem  3:  Let  c eR®,  c > 0,  P * = {X  | X >0,  AX  < b + ac},  a>0,  a «R,  and 
suppose  that  PQ  ^ 0,  and  PQ  is  nonempty  and  bounded. 

Then  there  exist  X e P * such  that  v(D. ) > L iff 

Or  A 


Of  > sup 
w>0 


L - G(w)  - wb 
wc 


The  proof  is  essentially  the  same  as  that  for  Theorem  2,  which  is  a 


special  case. 

Theorem  2 for  a = 1 can  be  rephrased  in  terms  of  conjugate  functions, 
if  we  restrict  G to  the  domain  {w  j w > 0},  and  P = [X  > 0 | AX  < b}  is 
nonempty  and  bounded.  Indeed,  by  Theorem  2,  there  exists  X cP  with 


v(D.  ) > L if  and  only  if 
K 


(12a) 


1 > (L  - GCwjVw1!)  for  all  w > 0, 


or,  equivalently, 

(12b)  wHi  > L - G(w)  for  all  w > 0,  with  w^  > 0; 

and 

(12c)  L - G (w)  < 0 for  all  w > 0,  with  w^  = 0. 

Now  (12)  is  in  turn  equivalent  to  w^b  > L -G(w)  for  all  w > 0,  i.e., 

(13)  G(w)  - wt(-b)  > L for  all  w > 0. 

Finally,  (13)  is  equivalent  to -GC(-b)  > L,  where  GC  is  the  conjugate 
function  for  G [ 7 ] • 


4 


r 
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We  now  give  some  corollaries  of  the  previous  results,  where  the  cases 
(a)  and  (b)  referred  to  are  those  of  the  introduction. 

Corollary  1:  There  exist  \ > 0 such  that  v(D^)  > L and  (a)  is  satisfied 

iff  L < inf  {Me  + h(e)},  where  h(e)  = inf  {f  (x)  | g (x)  < e for  1 < i < k and  xeS}. 
e>0  1 “ 

Proof : We  take  A to  be  the  lxk  matrix  of  ones  and  apply  Theorem  1. 

For  any  utR  let  a = max  [Q,  max  £u^  | 1 < i < k}J.  Then  V(u)  = Mar  and 

P(u)  > p(a,...,ar)  so  inf  [v(u)  + p(u)}  = inf  {Me  + h(e)}. 


We  note  that  Corollary  1 also  iollows  directly  from  Theorem  2,  since, 
for  a lxk  matrix  of  ones,  C - h where  G is  defined  in  (10)  and  h is 
defined  in  Corollary  1.  Then  the  condition  w^  > L-G(w)  for  all  w > 0, 
of  the  discussion  following  Theorem  2,  becomes  w^M  > L-h(w^)  for  all 
w^  = s > 0,  and  we  obtain  Corollary  1. 

Corollary  2:  There  exist  \ such  that  v(D  ) > L and  (b)  is  satisfied  iff 

A. 

L < inf  {m(2u.)  + p (u) } . 
u >0 


k 

Proof : Here  A is  an  identity  matrix  and  V(u) =M  Z max  {u  ,0}.  If 

i-1 

u^  = max  {u^,0},  V(u#)  = V(u)  and  p(u' ) < p(u),  so  we  need  only  look  at 

those  u > 0 in  taking  inf  £v(u)  + p(u)}. 

u 

Q.E.D. 


Corollary  2 can  also  be  obtained  directly  from  Theorem  2;  we  omit  the 
proof . 

To  give  an  application  of  these  results  in  a computational  setting, 

we  take  Theorem  2 for  the  case  of  one  row  (m  = l);  in  particular,  we 

focus  on  the  case  (a)  of  the  introduction  for  M=1  (though  the  case  of 
k 

a constraint  Z a X < 1,  for  all  a > 0,  is  similarly  treated).  Since 
1 1 1 1 

a perturbation  function  h(c)  of  Corollary  1 is  involved  in  obtaining 
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upper  bounds  on  v(D  ),  which  already  embodies  a significant  amount  of 

A. 

information  on  the  primal  problem  (P)  (note  that  h(0)  alone  is  the  value 
of  (P)),  our  bounds  will  necessarily  require  some  knowledge  of  the  primal 
problem.  It  turns  out,  however,  that  to  get  bounds  on  h^(A,L)  of 
Theorem  2 in  this  case,  does  not  require  a knowledge  of  all  of  h,  but 
only  of  h(e^)  and  h^)  for  values  0 < < 62*  The  analysis  follows 

in  the  next  paragraphs;  it  is  straightforward. 

One  trivially  proves  that  G as  defined  in  (10)  is  convex  on  its 
domain.  By  the  paragraph  following  vxuollary  1,  so  is  h of  Corollary  1. 
Now  assume  that  h(0)  is  defined  (i.e.,  (P)  has  finite  value  v(P))  and 
h(e)  is  defined  for  at  least  one  e > 0.  (From  Theorem  2,  if  h(e)  =-« 

for  any  e > 0,  then  v(D.  ) = - « for  all  X > 0.)  By  convexity,  h(e)  is 

A. 

then  defined  for  all  e > 0,  and  continuous  for  e > 0;  clearly,  h is 
nonincreasing  in  e > 0.  Therefore,  the  graph  of  h looks  like  the 
function  drawn  in  Figure  1. 

In  Figure  1,  we  have  drawn  the  case  where  L#  = sup  £h(c)  | c > 0} 

< h(0)  = v(P),  which  can  occur  if  h is  discontinuous  at  its  boundary 
point  e = 0 (of  course,  L#  = h(0)  = v(P)  is  forced  if  there  is  a Slater 
point  for  (P),  for  then  h(c)  is  defined  for  some  e < 0 and  0 is  an 
interior  point  of  the  domain  of  h,  hence  a point  of  continuity).  This 
need  not  be  the  case;  if  h is  continuous  at  e =0,  L#  = h(0)  = v(P). 

Also  in  Figure  1,  we  have  h 1 (0)  = - ® , where  h*  (0)  indicates  the  right 
directional  derivative  of  h at  0,  which  exists  as  h is  convex.  This 
also  need  not  be  the  case,  for  1/(0)  > - <»  is  possible,  and  then  (as 
l/  (0)  < 0 by  monotonicity)  h/ (0)  would  be  finite.  Thus  there  are 
actually  four  possibilities  at  e = 0 in  drawing  the  graph  of  h 
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upper  bounds  on  v(D^),  which  already  embodies  a significant  amount  of 
information  on  the  primal  problem  (P)  (note  that  h(0)  alone  is  the  value 
of  (P)),  our  bounds  will  necessarily  require  some  knowledge  of  the  primal 
problem.  It  turns  out,  however,  that  to  get  bounds  on  h^(A,L)  of 

Iheoreni  2 in  this  case,  does  not  require  a knowledge  of  all  of  h,  but 

only  of  h(e^)  and  h^)  for  values  0 < < 62*  The  analysis  follows 

in  the  next  paragraphs ; it  is  straightforward . 

One  trivially  proves  that  G as  defined  in  (10)  is  convex  on  its 

domain.  By  the  paragraph  following  ooiollary  1,  so  is  h of  Corollary  1. 

Now  assume  that  h(0)  is  defined  (i.e.,  (P)  has  finite  value  v(P))  and 
h(e)  is  defined  for  at  least  one  e > 0.  (From  Theorem  2,  if  h(e)  = - «° 
for  any  e > 0,  then  v(D^)  = - <*>  for  all  X > 0.)  By  convexity,  h(e)  is 
then  defined  for  all  e > 0,  and  continuous  for  e > 0;  clearly,  h is 
nonincreasing  in  e > 0.  Therefore,  the  graph  of  h looks  like  the 
function  drawn  in  Figure  1. 

In  Figure  1,  we  have  drawn  the  case  where  L#  = sup  {h(c)  | e > 0) 

< h(0)  = v(P),  which  can  occur  if  h is  discontinuous  at  its  boundary 

point  e = 0 (of  course,  L#  = h(0)  = v(P)  is  forced  if  there  is  a Slater 

point  for  (P),  for  then  h(e)  is  defined  for  some  e < 0 and  0 is  an 

interior  point  of  the  domain  of  h,  hence  a point  of  continuity).  This 
need  not  be  the  case;  if  h is  continuous  at  e = 0,  L#  = h(0)  = v(P). 

Also  in  Figure  1,  we  have  h7  (0)  = -<*>,  where  h7  (0)  indicates  the  right 
directional  derivative  of  h at  0,  which  exists  as  h is  convex.  This 
also  need  not  be  the  case,  for  h7  (0)  > - <=°  is  possible,  and  then  (as 
h7 (0)  < 0 by  mono tonicity)  h7 (0)  would  be  finite.  Thus  there  are 
actually  four  possibilities  at  e = 0 in  drawing  the  graph  of  h 
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L 

(corresponding  to  L#  < v(P)  or  L#  = v(P),  and  h7  (0)  = - « or  h'  (0) 

of  which  we  have  drawn  only  one.  Furthermore,  in  our  picture  h has  a 

horizontal  asymptote,  which  simply  means  that  the  criterion  function  f 

of  (P)  is  bounded  below;  but  this  need  not  be  the  case  in  general. 

Note  that  P , as  defined  above  Theorem  2,  is  always  nonempty  for  all 

a > 0,  for  the  constraint  (a)  with  M = l.  By  Theorem  2,  there  exists  X 
k 

with  X > 0 and  2 X^  < a if  and  only  if  a > h^ (A,L) . The  supremum  quotient 
on  the  r.h.s.  in  (11)  has  an  interesting  interpretation;  here  w = (w^)  and 
b = 0^)  = (1). 

For  this  quotient  to  be  finite,  one  certainly  requires  (L-G(0))/(0  * 1) 
= (L-h(0))/0  <+oo,  which,  by  the  conventions  in  Theorem  2,  necessitates 
L < h(0)  = v(P).  This  fact  is  well  known:  the  dual  value  v(D)  cannot 
exceed  v(P). 

Moreover,  in  addition  to  L < v(P),  for  this  quotient  to  be  finite 
for  L = L#,  requires  that  the  right-sided  directional  derivative  g7 (e) 


of  the  function  g(e)  be  finite  at  zero,  where  g is  the  continuous  convex 
function  defined  by: 

f L#,  if  e ■ 0; 


(14)  g(e)  - ( 

I h(e) , if  e > 0. 

Indeed,  the  supremum  quotient  on  the  r.h.s.  in  (11),  for  w * (w^)  > 0, 
is  simply  the  negative  of  this  right-sided  directional  derivative.  Thus, 
when  (as  in  Figure  1)  this  directional  derivative  is  -<*>,  one  has 

v(D.  ) < L (A  for  all  X > 0 (as  a - + » is  forced,  by  Theorem  2);  when  this 

A. 


directional  derivative  is  finite,  there  is  X > 0 with  v(D^ ) * L#  and  in 
k X 

fact  2 X^  < -g7 (0)  = h,(A,L#)  (by  Theorem  2).  By  the  same  reasoning, 
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one  cannot  have  v(D^)  > L for  X > 0,  if  L > L#,  for  then  the  r.h.s.  in 
(11)  must  be  unbounded:  this  is  a consequence  of  the  well-known  fact 
that  lim  {pUj, . . . , e^)  | . . . ,e^\0+}  is  the  value  of  the  dual  in  (P). 

Now  the  analysis  of  the  previous  paragraph  extends  to  a point 
c*  > 0,  for  this  yields  a convex  program  when  translated  to  the  origin 
(using  convex  constraints  g (x)  - e*  < 0,  for  1 < i < k).  Thus,  the 
negative  of  the  right-sided  directional  derivative  of  g at  e «*  e* 

k 

(which  is  that  of  h,  by  continuity  at  e = e*)  is  a lower  bound  on  E X. 

1 1 

for  the  translated  program.  By  convexity  of  h,  this  directional 

derivative  is  not  less  than  the  directional  derivative  at  e = 0.  Hence, 
k 

a lower  bound  on  E X.  for  (P)  is  -h(e*)  for  any  e*  > 0.  (Since  h(e)  is 
1 1 

defined  also  for  0 < e < e*,  the  same  is  true  of  the  left-sided 

directional  derivative,  by  convexity  of  h,  and  the  left-sided  derivative 

of  course  gives  a better  lower  bound,  in  Figure  1,  we  have  drawn  the 

case  that  h is  differentiable  at  e * e*.) 

Now  the  directional  derivative  of  a perturbation  function  can  be 

hard  to  compute,  but  again  the  convexity  of  h simplifies  the  matter. 

In  fact,  if  we  know  h(e1)  and  h(«2)  for  two  values  0 < < e2,  then  a 

k 

lower  bound  on  I X is  given  by  the  negative  of  the  slope  of  the  secant 
1 1 

line  CD,  to  whit. 


(15) 


(h(«j_)  - h(e2))/(e2  - ex). 


Indeed,  this  secant  line  has  slope  not  less  than  ^(e,),  and  -h' (c, ) is 
k 11 

a lower  bound  on  Z X. . Clearly,  the  lower  bound  given  by  (15)  improves 
1 1 

as  nears  zero,  and  e2  > is  nearer  to  . 

k 

However,  all  the  quantities  (15)  are  lower  bounds  on  E . If, 

1 

in  a computational  setting,  any  of  these  bounds  (15)  is  excessively  large, 


one  knows  that  the  theoretically  optimal  dual  value  L#  = v(0)  cannot  be 
computationally  attained,  and  one  will  have  to  settle  for  a lower  value 
L < L#. 

For  L < L# , the  supremum  quotient  in  equation  (11)  clearly  gives 

the  negative  of  the  slope  of  a line  through  (0,L)  that  is  tangent  to 

the  graph  of  h(*)«  or,  lacking  such  a point  of  tangency,  asymptotic  to  it 

or  parallel  to  an  asymptote  of  it.  Clearly,  if  the  directional  derivative 

at  e - *2  > 0 intersects  the  vertical  axis  in  Figure  1 at  a point  not  below 

L,  then  «2  is  at  or  to  t*ie  of  this  point  of  tangency,  if  it  exists  (if 

an  asymptote  exists,  we  vie*.:  all  e > 0 as  "to  the  left  of"  the  "point  of 

k 

tangency").  Hence  an  upper  bound  on  £ X^  to  attain  v(D^)  > L is  -h* (62)* 

so  another  (generally  weaker)  upper  bound  is  given  by  (15).  Thus,  if  the 

value  of  (15)  is  computationally  feasible,  it  yields  a number  L such  that 

k 

v(D,  ) > L for  some  X > 0 with  E X.  computationally  feasible. 

K j 1 

Section  2:  {X  | AX  < b,  X > 0)  may  be  unbounded 


One  way  of  approaching  the  case  treated  in  this  section  is  by  a simple 
reduction  to  the  bounded  case  of  the  previous  section.  Again 
P = {X  J AX  < b,  X > 0},  V(u)  is  as  defined  in  equation  (3),  and  p(u)  is 
defined  as  in  equation  (4).  We  shall  also  need  the  notations: 


(16)  P'  - [X  I AX  < b,  X > 0,  E X.  < M} 
M - i 1 


(17)  V^(u)  " max  I X ePM^' 


Here  is  a result  one  can  obtain  as  an  lnraediate  application  of 


Theorem  1. 
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Corollary  3:  Suppose  that  P i 0,  and  set 

(18)  L*  “ 11m  Inf,  [v' (u)  + p(u)}. 

M-+oo  utR 


If  L*  cRU  then  v(Dx)  < L*  for  all  X with  X > 0 and  AX  < b. 

Moreover,  if  L*  e R and  i/  < L*,  there  exists  X > 0 with  AX  < b and 
v(Dx)  > i/  . 

Finally,  I,*  eRU  {-  » }. 

/ k 

Proof:  Clearly,  VM(u)  is  monotone  nondecreasing  in  M for  any  ueR  , 

hence  the  same  is  true  of  inf  (v'(u)  + p(u)}.  Thus,  the  limit  indicated 

ueR* 

in  (18)  exists. 

If  L*  =-»,  then  by  monotonicity  inf,  {v'(u)  + p(u)}  = 

_K  M 
ueR 

for  all  M > 0.  If  X > 0 satisfies  A\  < b,  then  X e P*  for  sufficiently 

M 

large  M,  and  Theorem  1 applies,  showing  v(D.  ) = - °o. 

A. 

If  L*eR,  let  L#  > L*.  By  monotonicity,  inf,  { v'(u ) + p(u)}  < L# 

n K M 
ueR 

for  all  M.  If  X > 0 satisfies  AX  < b,  then  X e P^.  for  sufficiently 

M 

large  M,  and  by  Theorem  1 we  have  v(P  ) < L# . Since  v(P  ) < L#  holds 

• A A 

for  all  L#  > L*,  actually  we  have  v(P  ) < L*  also. 

A 

Next,  if  i/  < L*,  by  monotonicity,  for  sufficiently  large  M, 

inf  {v^(u)  + p(u)}  > l'  . By  Theorem  1,  for  Mn  sufficiently  large, 
u*Rk  M 0 

there  is  X e P^,  C P with  v(D.  ) = inf  {v' (u)  + p(u)]  > l'  . 

" M0  ^ ueRk  “0 

Finally,  by  the  last  paragraph,  if  l*  = + ® then  v(D^)  > v(P)  for 

some  X eP,  since  (P)  is  assumed  consistent.  However,  as  v(D)  < v(P), 

and  v(D)  = sup  {v(Dx)  | X > 0},  we  have  a contradiction. 

Q.E.D. 


! 


! 


... 


L 
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Hie  proof  of  Corollary  3 makes  it  evident,  that  there  is  X e P with 
v(D^)  - L*  if  and  only  if  the  limiting  operation  in  (18)  is  vacuous,  i.e., 
if  and  only  if 

(19)  L*  - inf  £v' (u)  + p(u)3 

ueR 

for  all  sufficiently  large  M. 

It  is  clear  that 

(20)  Vjj(u)  + P(u)  < V(u)  + p(u) 

k 

for  all  ueR  , where  one  may  interpret  ® - » on  the  r.h.s.  of  (20)  to  be  - • . 
Taking  the  inf imum  over  ueRK  and  then  applying  the  limiting  operation  to 
both  sides  of  (20)  yields 

(21)  L*  < infk  {V(u)  + p(u)). 

ueR 

However,  unlike  the  bounded  case,  strict  inequality  can  occur  in  (21),  i.e., 
one  can  have 


(22)  sup  £v(D.  ) | X c P}  < infk  £v(u)  + p(u)} 

ueR 

as  our  next  example  shows . 


Example.  Let  P = {X  | X > 0j.  Then  we  have  for  u eR  , u = (u^,^). 


(23) 

and 

(24) 


V(u)  = 


0 , if  u^  < 0 and  Uj  < 0; 
+ ® , if  u^  > 0 or  U2  > 0; 


0 , if  u^  < 0 and  Uj  < 0; 

M max  Cu^.Uj},  if  u^  > 0 or  U2  > 0. 
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We  calculate,  using  the  fact  that  p(u)  is  monotone  non- increasing  In  u, 
inf  {v(u)  + p(u)}  =■  inf  {p(u)  | ^ < 0 and  u2  < 0) 


provided  that  p(u)  is  never  - ® . 

Now  p(0)  is  the  value  v(P)  of  the  primal  problem  (P),  while 
v(D)  * sup  {v(D^)  | \ > 0]  is  the  value  of  the  dual.  As  is  well-known, 
one  can  have  v(D^)  < v(P)  even  if  p(u)  is  never  - «° . 

For  example,  in  R with  S = { (w^  ,w2)  | Wj.  > 0 and  w2  > 0],  f(x)  * f(w^,w2) 
- max  {-1,-  Vwiw2  3 (recall  that  V wiw2  is  conca-ve),  k = 1,  g^x)  = g1(w1,w2) 

* w^,  we  have  v(P)  = p(0)  = 0.  However,  p(u)  = inf  {max  {-1,-^/wjw^  } | w^  < u} 
is  such  that  p(u)  = + ® if  u < 0,  p(u)  = -1  if  u > 0 (since  - uw2  < -1  for 
w2  > 0 sufficiently  large,  whenever  u > 0).  Therefore  v(D)  = lim  {p(u)|uiO+} 

= -1  < v(P)  (or  one  may  demonstrate  that  v(D)  * -1  directly).  In  this 
example,  all  functions  are  continuous  on  S. 

Without  introducing  some  form  of  "constraint  qualification"  which 
rules  out  examples  of  the  type  just  given,  there  is  another  result,  beyond 
that  of  Corollary  3,  which  involves  a different  analysis.  First  we  need  a 
preliminary  lemna  on  semi-infinite  systems  of  linear  inequalities. 


Lemma  2:  Let  a*  cRn,  b^  eR  for  i «I,  I + 0.  If  the  semi-infinite  system 
(26)  a Sc  > b , id 

is  inconsistent,  and  F S I is  a finite  subset  of  I,  there  exists  c *Rn,  d eR 
such  that  cx  > d is  obtained  as  a nonnegative  linear  combination  of  the 
inequalities  a Sc  > b^,  i |F  (plus  possibly  a decrease  in  the  right -hand -side) , 
with  the  property  that  the  semi-infinite  system 
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! 

(26/  a*x  > b , i«I\F 

cx  > d 

Is  also  Inconsistent. 

In  particular,  for  F fixed  and  finite,  F c I,  if  every  system  of  the 
form  (26 / is  consistent,  with  cx  > d obtained  as  described,  then  (26)  is 
also  consistent. 

Proof : Let  Qx  = [x  | aix  > b^  i eF}  and  let  Q2  « £x  | aAx  > b, , i c I\F}. 
Both  Qx  and  Q2  are  closed  convex  sets,  and  Q^CIC^  = 0 by  the  inconsistency 
of  (26). 

r 

Suppose  first  that  ^ 0.  Let  cx  > d be  a hyperplane  which  strictly 

separates  from  Q2>  i.e.,  cx  > d for  xcQ^  and  cx  < d for  x «Q2.  Then 

since  a^x  > b^,  i eF,  implies  cx  > d and  Q t 0,  cx  > d is  a nonnegative 

combination  of  the  inequalities  of  Q^,  with  possibly  a decrease  in  the 

r.h.s.  d.  Since  cx  < d for  all  xcQ2>  (26 / is  inconsistent. 

Next,  suppose  = 0.  Then  0-x  > 1 is  obtainable  as  a nonnegative 

combination  of  the  constraints  of  Q.  . Take  c ■ 0,  d = 1 in  (26/,  and 

*■ 

note  that  (26/  is  clearly  inconsistent. 

Q.E.D. 

The  following  is  a version  of  the  alternative  result  that  we  mentioned 
above  Lemma  2;  we  will  rephrase  it  in  final  form  as  Theorem  4 below 
(compare  Lemma  3 with  Lemma  1). 

Lemma  3:  Suppose  that  P ^ 0. 


There  is  no  X cF  with  v(D^)  > L if  and  only  if  there  is  auiR  with 
V(u)  < + ® such  that  for  every  M,  there  is  an  x such  that  the  finite  system 


'x.M.u 


uX  < V (u) 


is  inconsistent. 


S g.(x)X.  > L - f(x) 
i-1  1 L 


E X < M 
i°l  1 


X > 0 


Proof:  The  existence  of  X *?  such  that  v(D^)  > L is  precisely  equivalent 
to  the  consistency  of  the  semi -infinite  system 


AX  < b 


E g, (x)X . > L - f(x),  x c S 
i=l  1 


X > 0. 


By  Lemma  2,  (28)  is  inconsistent  if  and  only  if  there  exists  a vector 
of  multipliers  9 > 0 such  that 


GAX  < 9b 


(28/ 


E g . (x)X . > L - f (x) , x e S 
i-1  1 x 


X > 0 


is  inconsistent.  Put  u ■ 9A;  since  V(u)  < 9b,  we  see  that  V(u)  < + » , 
and  (28/  remains  inconsistent  with  V(u)  replacing  9b. 

Thus,  for  any  M,  the  following  semi-infinite  system  is  inconsistent: 


4 
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(28 )&  uX  < V(u) 

k 

E g* (x)X . > L - f (x) , x < S 
1-1 

k 

E X.  < M 

1-1 

X > o. 


k 

Since  the  polyhedron  Q__  — £X  > 0 | E X.  < M,  uX  < V(u)}  Is  either  empty 

" 1-1  1 

or  nonempty  and  bounded,  by  Lemma  1 (with  P replaced  by  Q^)  there  Is  an  x 


such  that  (27)  u 


Is  Inconsistent. 


Q.E.D. 


As  preliminary  remarks  to  Theorem  4 Just  below,  we  note  that  always 
V(u)  > - oo  since  P i 0.  Also,  V(u)  is  convex,  as  Is  well-known  (for 
example,  note  that  V(u)  - min  {9b  | 9A  > u,  9 > 0}  if  V(u)  < + ® , from 
duality;  and  a minimizing  linear  program  is  convex  in  Its  r.h.s.  u). 
Clearly,  V(<*u)  = o?V(u)  for  all  a > 0,  when  V(u)  < + * . 

Lenina  3 can  be  rephrased.  In  terms  of  the  perturbation  function  p, 
as  follows. 


Theorem  4:  Suppose  that  P t 0.  Define 

(29)  J - inf.  lim  Inf  (<W(u)  + p(oru  + pe)} 

usR  p-0  er  > 0 
V(u)  <+• 

where  e - (1,...,1)T  is  a vector  of  one's.  Than: 

(30a)  If  L < J,  there  exists  X cP  with  v(D^)  > L. 

(30b)  If  L > J,  there  does  not  exist  X «P  with  v(D^)  > L. 


i.e.,  J - L*,  where  L*  is  given  in  equation  (18). 


Proof : We  first  prove  (30a).  Fix  ueR  with  V(u)  <+  °°  . Define  the 
function 

(31)  h (P  ) = inf  {aV (u)  + p (cm  + p e ) ] . 

U a>0 

We  note  that  hu  is  a convex  function  of  p . In  fact , let  P j > P 2 > 0 be 
given  and  let  p^^  > hu(P^)f  p2  > hu<P2).  Then  there  exists  a^,  > 0 with 

(32a)  PL  > df^u)  + p(o'1u  + p Le) 

(32b)  P2  > a2V(u)  + p(a2u  + P2e). 

Now  if  0 < X < 1,  using  the  convexity  of  V and  p,  we  obtain  from  (32) 

(33)  \Pl+(l-X)p2  > XV^u)  + (1  -X)V(a2u) 

Xp(aLu  + p Le)  + (1  - X)p(a2u  + p2e) 

> V((Xox  + (1  -X)a2)u) 

+ P((Xax  + (1-X)a2)u+  (Xp  x + (1-X)|32)e) 

> inf  {V(au)  + p(au  + (Xp . + (1-X)p,)e)3 

a>0  1 z 

- hu(XPi  + (1  -x)p2). 

Since  p^  > bu(p^)  and  p2  > hy(P2)  were  arbitrary,  from  (33)  we  obtain 
Xhu0i>  + (1  *X)hu(p2)  > hu (Xp  ^ + (1  -X)p2)  and  our  Proof  of  the  convexity 
of  hu(*)  is  complete. 

Let  L < J.  Then,  by  the  definition  of  J as  the  r.h.s.  of  (29),  also 

L < lim+  hu(P).  Using  the  convexity  of  hu(*)»  hu<JJ  ) > L - MP  for  all 

p\o 

P > 0,  provided  only  that  M is  chosen  sufficiently  large  (M  need  only  not 
be  less  than  the  negative  of  the  slope  of  the  line  through  (0,L),  which 
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is  either  tangent  or  asymptotic  to  the  curve  given  by  v = h (0 ) for 

u k 

0 > 0).  Since  P # 0,  we  can  also  assume  that  there  is  X e P with  £ < M. 

1 

We  now  prove  that,  for  any  xeS,  the  system  (27 M u is  consistent. 

Then  by  Lemma  3,  there  is  X e P with  v(D.  ) > L and  we  have  (30a). 

A 

Now  if  (27)  is  inconsistent,  let  a > 0 be  the  multiplier  on  the 

x,M,u 

first  row,  6 > 0 the  multiplier  for  the  second  row,  and  0 > 0 the 
multiplier  on  the  third  row,  such  that,  by  the  Kuhn-Fourier  Theorem  [8], 


(34a) 


cm  - 6(g1(x),-..,gk(x))  + 0e  > 0 


(34b) 


aV(u)  + 6(f(x)  - L)  + 0M  < 0. 


In  (34),  we  must  have  6 > 0,  since  the  system  (27)  , with  the  second  row 

k x,M,u 

deleted,  is  consistent  (let  x e P be  such  that  £ \.  < M;  then  X solves  this 

1 1 

system).  Without  loss  of  generality,  6=1. 


Now  (34a)  becomes 


(g1(x),...,gk(x))  < oru  + 0e 


so  that  f(x)  > p(oru  + 0e).  Combining  this  with  (34b),  we  obtain 
(36)  otV (u)  + p(cm  + 0e)  < L - 0M 

from  which  follows  h^O)  < L - 0M.  However,  from  the  choice  of  M, 

h (8  ) > L - 0M,  and  we  have  a contradiction.  Thus  (27)  cannot  be 

U - x,M,u 

inconsistent. 

We  next  prove  (30b).  First  assume  J > - °°  . Let  L > J,  and  choose 

n so  that  L > J + 3/n.  Then  for  some  u eR  with  V(u)  < +«>  , we  have 

lim  h (0  ) < J + 1/n.  Let  M be  arbitrary.  For  all  0 sufficiently  small, 
0\  0+  u 

there  exists  an  a > 0 with 


I 
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(36) 


orV(u)  + p(cm  + 0e)  < J + 2/n. 


By  choosing  0 so  small  that  0M  < 1/n,  we  have 
(37)  aV(u)  + p(oru  + Pe)  + pM  < L. 

From  (37),  there  exists  a vector  xeS  with 


(38a) 

(38b) 


(g1(2S)  » ••  • >8k(x))  < au  + pe 
CW(u ) + (f  (x)  - L)  + PM  < 0. 


Now  (38)  shows  that  (27)  is  inconsistent.  Since  M was  arbitrary, 

x,M,u 

Lemma  3 shows  that  v(D  ) > L is  impossible  for  X eP. 

A. 

In  the  case  J = - ®,  (37)  is  easily  obtained,  and  the  same  argument 
will  work. 

Q.E.D. 

Our  next  and  final  characterization  of  sup  {v(D^)  | X eP}  for  the 
case  of  P / 0 and  P possibly  unbounded,  is  a direct  application  of 
Theorem  3.  First  we  define  the  altered  perturbation  function,  depending 
on  A,  by: 

(39)  H(w’wnri-l)  * inf  I sublect  to  x«s  and  (SiOO gk^X^ 

* V + Wmfle3 

where  (w.w^  j^)  « R1^ 1 , wcR™,  and  e = (1,...,1)T  is  again  the  vector  of 
one's  (compare  (39)  with  (10)).  Now  (39)  is  easily  seen  to  be  the 
previous  perturbation  function  of  type  (10)  for  the  constraint  system 


(40) 


+ or 


in  which  a row  E X,  < 1 has  been  added  to  the  constraints  of  P to 
1 


obtain  a bounded  polyhedron,  and  y cR. 
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Theorem  5:  Suppose  that  P ^ 0 and  b > 0. 

There  exists  X e P such  that  v(D  ) > L if  and  only  if  the  supremum 

A 

(41)  a = sup  (L  - H(w,wnfl)  - , 

is  finite.  When  this  supremum  is  finite,  in  fact  there  is  X e P with 
k k 

v(D^ ) > L and  E ^ = ct;  and  there  is  no  X e P with  v(D^)  > L and  E <ct. 

Proof:  We  apply  Theorem  3 with  replacing  b,  and  with  replacing  c, 

where  y = min  {E  \.  | X c P}. 

1 

Q.E.D. 

It  is  worth  noting  that,  in  Theorems  4 and  5 (the  latter  with  H as 
defined  in  (39)),  the  vector  e can  be  replaced  by  any  vector  with  all 
components  strictly  positive.  In  fact,  the  same  proofs  are  valid  with 
this  change. 
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